Information processing apparatus and non-transitory computer readable medium

ABSTRACT

An information processing apparatus includes a communication unit, a detection unit, and a remote operation unit. The communication unit communicates with an external device. The detection unit detects a sound. The remote operation unit operates the external device with at least one of operations on the external device invalidated, and validates an operation on the external device, which has been invalidated, in the case where a first specific sound is detected by the detection unit with a reception screen for receiving a specific input by a user displayed on the external device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 fromJapanese Patent Application No. 2016-126784 filed Jun. 27, 2016.

BACKGROUND Technical Field

The present invention relates to an information processing apparatus anda non-transitory computer readable medium.

SUMMARY

According to an aspect of the present invention, there is provided aninformation processing apparatus including: a communication unit thatcommunicates with an external device; a detection unit that detects asound; and a remote operation unit that operates the external devicewith at least one of operations on the external device invalidated, andthat validates an operation on the external device, which has beeninvalidated, in a case where a first specific sound is detected by thedetection unit with a reception screen for receiving a specific input bya user displayed on the external device.

BRIEF DESCRIPTION OF THE DRAWINGS

An exemplary embodiment of the present invention will be described indetail based on the following figures, wherein:

FIG. 1 is a block diagram illustrating an image forming system accordingto an exemplary embodiment of the present invention;

FIG. 2 is a block diagram illustrating an image forming device accordingto the exemplary embodiment;

FIG. 3 is a block diagram illustrating a terminal device according tothe exemplary embodiment;

FIG. 4 illustrates an example of a general menu screen;

FIG. 5 illustrates an example of a function selection screen;

FIG. 6 illustrates an example of a copy setting screen;

FIG. 7 is a sequence diagram illustrating an example of a processperformed by the image forming system according to the exemplaryembodiment;

FIGS. 8A to 8C illustrate an example of screen transition in theterminal device;

FIGS. 9A to 9C illustrate an example of screen transition in the imageforming device;

FIGS. 10A to 10C illustrate an example of screen transition in the imageforming device;

FIGS. 11A to 11C illustrate an example of screen transition in theterminal device; and

FIGS. 12A to 12C illustrate an example of screen transition in the imageforming device.

DETAILED DESCRIPTION

An image forming system that serves as an information processing systemaccording to an exemplary embodiment of the present invention will bedescribed with reference to FIG. 1. FIG. 1 illustrates the image formingsystem according to the exemplary embodiment. The image forming systemincludes an image forming device 10 and a terminal device 12. The imageforming device 10 and the terminal device 12 communicate with each othervia a communication path N such as a network. In the example illustratedin FIG. 1, one image forming device 10 is included in the image formingsystem. However, plural image forming devices 10 may be included in theimage forming system. As a matter of course, other devices may beincluded in the image forming system.

The image forming device 10 is a device that includes an image formingfunction. Specifically, the image forming device 10 is a device thatincludes at least one of a scan function, a print function, a copyfunction, and a facsimile function. The image forming device 10 alsoincludes a function of transmitting and receiving data to and from otherdevices.

The terminal device 12 is a device such as a personal computer (PC), atablet PC, a smartphone, and a cellular phone, and includes a functionof transmitting and receiving data to and from other devices.

In the image forming system according to the exemplary embodiment, theterminal device 12 includes a function of remotely operating the imageforming device 10. In this case, the terminal device 12 corresponds toan example of the information processing apparatus, and the imageforming device 10 corresponds to an example of the external device. As amatter of course, the image forming device 10 may include a function ofremotely operating the terminal device 12. In this case, the imageforming device 10 corresponds to an example of the informationprocessing apparatus, and the terminal device 12 corresponds to anexample of the external device.

A case where the terminal device 12 has the function of remotelyoperating the image forming device 10 will be described below. Theterminal device 12 is provided at a customer center for the imageforming device 10, and used by an operator at the customer center (e.g.a person who manages the image forming device 10), for example. It isassumed that a user (e.g. a customer) of the image forming device 10 andthe operator at the customer center communicate with each other bytelephone, through a video conference system, or the like, and that theoperator explains a method of operating the image forming device 10 tothe customer or instructs the customer how to operate the image formingdevice 10, for example. It is assumed that the operator remotelyoperates the image forming device 10 using the terminal device 12 insuch situations.

The configuration of the image forming device 10 will be described indetail below with reference to FIG. 2. FIG. 2 illustrates theconfiguration of the image forming device 10.

A communication section 14 is a communication interface, and includes afunction of transmitting data to other devices and a function ofreceiving data from other devices via the communication path N. Thecommunication section 14 may be a communication interface that includesa wireless communication function, or may be a communication interfacethat includes a wired communication function.

An image forming section 16 performs an image forming process. Forexample, the image forming section 16 executes at least one of a scanfunction, a print function, a copy function, and a facsimile function.When the scan function is executed, a document is read to generate scandata (image data). When the print function is executed, an image isprinted on a recording medium such as paper. When the copy function isexecuted, a document is read to be printed on a recording medium. Whenthe facsimile function is executed, image data are transmitted orreceived by facsimile. The image data may be image data generated byreading a document, or may be image data transmitted from other devices.A composite function obtained by combining plural functions may beexecuted. For example, a scan-transfer function obtained by combiningthe scan function and a transmission function (transfer function) may beexecuted. When the scan-transfer function is executed, a document isread to generate scan data (image data), and the scan data aretransmitted to a transmission destination. As a matter of course, such acombined function is merely exemplary, and different combined functionsmay be executed.

A storage section 18 is a storage device such as a hard disk. Thestorage section 18 stores execution instruction information (e.g. jobinformation) that indicates an instruction to execute the image formingprocess, image data to be printed, scan data generated by executing thescan function, various control data, various programs, terminal addressinformation that indicates the address of the terminal device 12,information that indicates the amount to be billed, the number of timesof reception of execution of a process corresponding to the billing,information on resources that have been used (e.g. the count of sheetsof paper), and so forth. As a matter of course, such information may bestored in separate storage devices, or may be stored in one storagedevice.

A user interface (UI) section 20 includes a display section and anoperation section. The display section is a display device such as aliquid crystal display. The operation section is an input device such asa touch screen and a keyboard, and receives various operations includinga remote operation.

A sound detection section 22 is a sound collector such as a microphone,and includes a function of detecting a sound. For example, when a user(e.g. a customer) who uses the image forming device 10 utters a voice,the voice is detected by the sound detection section 22.

A controller 24 controls operation of the various components of theimage forming device 10. For example, the controller 24 invalidates atleast one of operations on the UI section 20 under specific conditions.In the case where an operation is invalid, the operation may not bereceived. The controller 24 cancels the invalidation of operations onthe UI section 20 and validates the operations under other specificconditions. In the case where an operation is valid, the operation maybe received. For example, operations on the UI section 20 are validatedwhen a specific sound (corresponding to an example of the first specificsound) is detected in the terminal device 12, and at least one ofoperations on the UI section 20 is invalidated when another specificsound (corresponding to an example of the second specific sound) isdetected by the sound detection section 22 of the image forming device10. In addition, at least one of operations on the UI section 20 may beinvalidated when still another sound (corresponding to an example of thethird specific sound) is detected in the terminal device 12.

The configuration of the terminal device 12 will be described in detailbelow with reference to FIG. 3. FIG. 3 illustrates the configuration ofthe terminal device 12.

A communication section 26 is a communication interface, and includesfunctions of transmitting and receiving data to and from other devicesvia the communication path N. The communication section 26 may be acommunication interface that includes a wireless communication function,or may be a communication interface that includes a wired communicationfunction.

The storage section 28 is a storage device such as a hard disk, andstores various programs, various data, device address information thatindicates the address of the image forming device 10, and so forth.

A user interface (UI) section 30 includes a display section and anoperation section. The display section is a display device such as aliquid crystal display. The operation section is an input device such asa touch screen, a keyboard, and a mouse. The UI section 30 correspondsto an example of the operation reception unit.

A sound detection section 32 is a sound collector such as a microphone,and includes a function of detecting a sound. For example, when a user(e.g. an operator at the customer center) who uses the terminal device12 utters a voice, the voice is detected by the sound detection section32.

A controller 34 controls operation of the various components of theterminal device 12. The controller 34 includes a remote operationsection 36.

The remote operation section 36 remotely communicates with the imageforming device 10 via the communication path N, and includes a functionof remotely operating the image forming device 10. For example, theremote operation section 36 receives data on a screen being displayed onthe UI section 20 of the image forming device 10 from the image formingdevice 10, and causes the UI section 30 of the terminal device 12 todisplay the screen. Consequently, in a remote communication state, thescreen being displayed on the image forming device 10 is shared by theterminal device 12. The terminal device 12 may display the same screenas the screen being displayed on the image forming device 10, or maydisplay a screen prepared by customizing (such as modifying orprocessing) the screen being displayed on the image forming device 10for the terminal device 12. The remote operation section 36 alsocontrols the authority to operate the screen being displayed on the UIsection 20 of the image forming device 10. That is, the remote operationsection 36 transfers the operation authority from the image formingdevice 10 to the terminal device 12 under specific conditions, andtransfers the operation authority from the terminal device 12 to theimage forming device 10 under other specific conditions. On the deviceto which the operation authority has been transferred, that is, thedevice having the operation authority, an operation on the screen beingdisplayed on the UI section 20 of the image forming device 10 is valid.On the device from which the operation authority has been transferred,that is, the device not having the operation authority, an operation onthe screen is invalid. An operation may be received on the device havingthe operation authority, and an operation may not be received on thedevice not having the operation authority. In the case where theterminal device 12 has the operation authority, an operation on theimage forming device 10 through the UI section 30 of the terminal device12 is valid. In the case where the terminal device 12 does not have theoperation authority, an operation on the image forming device 10 throughthe UI section 30 of the terminal device 12 is invalid.

For example, in the case where a specific sound (corresponding to anexample of the first specific sound) is detected by the sound detectionsection 32 of the terminal device 12 when at least one of operations onthe UI section 20 of the image forming device 10 is invalid and aspecific reception screen is displayed on the UI section 20 of the imageforming device 10 in the remote communication state, the remoteoperation section 36 controls the image forming device 10 so as tovalidate the operations on the UI section 20 of the image forming device10 which have been invalid. In the case where the specific receptionscreen is not displayed on the UI section 20 of the image forming device10, the remote operation section 36 does not validate the operations onthe UI section 20 of the image forming device 10 which have beeninvalidated, even in the case where the specific sound (corresponding toan example of the first specific sound) is detected. The specificreception screen may be a screen that receives a specific input made bya user (e.g. a customer). Examples of such a screen include a screenthat prompts the user to make an input or an operation, morespecifically a function selection screen, a menu screen about the imageforming function, a setting screen about a specific image formingfunction (e.g. the copy function), a password input screen, and abilling screen (e.g. a screen that displays a function corresponding tospecific billing and a screen in which a billing process is performed byexecuting a function). The first specific sound may be a voice about atransfer of the operation authority from the terminal device 12 to theimage forming device 10. Examples of the first specific sound includevoices such as “Go ahead”, “Press”, and “Please”.

In the case where the specific sound (corresponding to an example of thefirst specific sound) is detected by the sound detection section 32 whenthe specific reception screen is displayed on the UI section 20 of theimage forming device 10 in the remote communication state, the remoteoperation section 36 invalidates an operation through the UI section 30of the terminal device 12, that is, prohibits reception of an operationthrough the UI section 30 of the terminal device 12. In addition, theremote operation section 36 may display, on the UI section 30 of theterminal device 12, information indicating that an operation on the UIsection 30 of the terminal device 12 is invalid. Examples of theinformation indicating that an operation is invalid include display of amessage saying “External device (image forming device 10) has operationauthority”, “Reception of operation is prohibited”, or the like. Theremote operation section 36 may display information on the UI section 30of the terminal device 12 when the UI section 30 of the terminal device12 receives an operation. When the UI section 30 of the terminal device12 receives an operation, the remote operation section 36 may displayinformation on the UI section 30 of the terminal device 12 and erase theinformation from the UI section 30, or may keep displaying theinformation, when a time of a threshold or more has elapsed, or may varythe manner of display of the information between immediately after theinformation is displayed and when a threshold time has elapsed (e.g.display characters in a larger size or in an otherwise conspicuousmanner immediately after the characters are displayed and display thecharacters in a smaller size or at a different location when a thresholdtime has elapsed) to display the information in a manner that isdistinguishable from the manner of display immediately after theinformation is displayed. In the case where the specific receptionscreen is not displayed on the UI section 20 of the image forming device10, the remote operation section 36 does not invalidate the operationsthrough the UI section 30 of the terminal device 12 but keeps theoperations valid, even in the case where the specific sound(corresponding to an example of the first specific sound) is detected.

When another specific sound (corresponding to an example of the secondspecific sound) is detected by the sound detection section 22 of theimage forming device 10 in the case where operations on the UI section20 of the image forming device 10 are valid in the remote communicationstate, the controller 24 of the image forming device 10 invalidatesoperations on the UI section 20 of the image forming device 10, and theremote operation section 36 validates operations on the UI section 30 ofthe terminal device 12. In addition, the remote operation section 36 maydisplay, on the UI section 30 of the terminal device 12, informationindicating that an operation on the UI section 30 of the terminal device12 is valid. Examples of the information indicating that an operation isvalid include display of a message saying “Operation authority has beentransferred to you”, “You have operation authority”, or the like. Forexample, when a second specific sound is detected by the sound detectionsection 22 of the image forming device 10 with information indicatingthat an operation on the UI section 30 of the terminal device 12 isinvalid displayed on the UI section 30 of the terminal device 12, theremote operation section 36 validates operations on the UI section 30 ofthe terminal device 12, and causes the display section of the UI section30 of the terminal device 12 to display information indicating that anoperation on the UI section 30 is valid. The second specific sound maybe a voice about a transfer of the operation authority from the imageforming device 10 to the terminal device 12. Examples of the secondspecific sound include voices such as “Operation is finished” and“Operation is completed”.

When still another specific sound (corresponding to an example of thethird specific sound) is detected by the sound detection section 22 withoperations on the UI section 20 of the image forming device 10 valid andwith operations on the UI section 30 of the terminal device 12 invalidin the remote communication state, the remote operation section 36 mayvalidate operations on the UI section 30 of the terminal device 12 andinvalidate operations on the UI section 20 of the image forming device10 which have been validated. The third specific sound may be a voiceabout a transfer of the operation authority from the image formingdevice 10 to the terminal device 12. Examples of the third specificsound include voices such as “Operator will operate”.

The image forming system according to the exemplary embodiment will bedescribed in detail below.

First, the screen displayed on the UI section 20 of the image formingdevice 10 will be described in detail. FIG. 4 illustrates a general menuscreen 38 that serves as an example of the screen. The general menuscreen 38 is a principal menu screen (e.g. a topmost screen), and thecontroller 24 of the image forming device 10 causes the UI section 20 todisplay the general menu screen 38 as the initial screen, for example.The general menu screen 38 includes, as displayed thereon, a buttonimage for designating the image forming function and button images fordesignating other services, for example. As a matter of course, theinformation displayed on the general menu screen 38 is merely exemplary,and the general menu screen 38 may include other information asdisplayed thereon, or a part of the information in the illustration maynot be displayed.

FIG. 5 illustrates a function selection screen 40 as another example ofthe screen. For example, when the button image for selecting the imageforming function is pressed by the user on the general menu screen 38,the controller 24 of the image forming device 10 causes the UI section20 to display the function selection screen 40. That is, the screentransitions from the general menu screen 38 to the function selectionscreen 40. The function selection screen 40 includes, as displayedthereon, button images for designating individual image formingfunctions (e.g. the copy function, the print function, the scanfunction, and the facsimile function). When a button image is pressed bythe user, the controller 24 of the image forming device 10 causes the UIsection 20 to display a screen for setting a function corresponding tothe button image. The information displayed on the function selectionscreen 40 is merely exemplary, and the function selection screen 40 mayinclude information about other functions as displayed thereon, or apart of the information in the illustration may not be displayed. Whenanother button image is pressed by the user on the general menu screen38, a screen corresponding to the button image is displayed on the UIsection 20.

FIG. 6 illustrates a copy setting screen 42 as another example of thescreen. When a button image corresponding to the copy function ispressed by the user on the function selection screen 40, for example,the controller 24 of the image forming device 10 causes the UI section20 to display the copy setting screen 42. That is, the screentransitions from the function selection screen 40 to the copy settingscreen 42. Setting items about copying are displayed on the copy settingscreen 42. Setting for copying is made on the copy setting screen 42.When an instruction for copying is provided by the user, the imageforming device 10 executes copying. In the case where a button imagecorresponding to an image forming function other than the copy functionis pressed by the user, a setting screen corresponding to the designatedimage forming function is displayed on the UI section 20, and settingfor the image forming function is made on the setting screen.

In the remote communication state, the screen (such as the general menuscreen 38, the function selection screen 40, and the copy setting screen42) displayed on the UI section 20 of the image forming device 10 isshared by the terminal device 12. That is, in the case where the screenis displayed on the UI section 20 of the image forming device 10, dataon the screen are transmitted from the image forming device 10 to theterminal device 12, and the remote operation section 36 of the terminaldevice 12 causes the UI section 30 of the terminal device 12 to displaythe screen. Consequently, the screen is shared by the image formingdevice 10 and the terminal device 12.

In the example described above, the general menu screen 38 correspondsto the principal menu screen (e.g. a topmost screen), and the functionselection screen 40 corresponds to a screen that is subordinate to thegeneral menu screen 38. However, the function selection screen 40 may bethe principal menu screen. In this case, the general menu screen 38 isnot displayed on the UI section 20, and the function selection screen 40is displayed on the UI section 20 as the initial screen.

A process performed by the image forming system according to theexemplary embodiment will be described below with reference to FIG. 7.FIG. 7 is a sequence diagram illustrating the process.

By way of example, it is assumed that the user (e.g. a customer) of theimage forming device 10 and the user (e.g. an operator at the customercenter) of the terminal device 12 are having a conversation with eachother by telephone or through a video conference system, and that theimage forming device 10 and the terminal device 12 are remotelycommunicating with each other. In addition, it is assumed that in theremote communication state, the terminal device 12 has the authority tooperate the UI section 20 of the image forming device 10 and the imageforming device 10 does not have the authority to operate the UI section20 of the image forming device 10 itself. That is, it is assumed thatoperations of the image forming device 10 through the UI section 30 ofthe terminal device 12 are valid, and that operations of the imageforming device 10 through the UI section 20 of the image forming device10 are invalid. A case where the operation authority is transferred fromthe terminal device 12 to the image forming device 10 in this state willbe described below.

With a specific reception screen (e.g. a menu screen, the functionselection screen 40, a password input screen, or a billing screen)displayed on the UI section 20 of the image forming device 10, the user(e.g. an operator at the customer center) of the terminal device 12utters a voice for transferring the operation authority to the imageforming device 10 (S01). For example, the operator utters a voice suchas “Go ahead”, “Press”, and “Please”.

When the sound detection section 32 of the terminal device 12 detects afirst specific sound (e.g. the voice such as “Go ahead” described above)(S02), the remote operation section 36 causes the UI section 30 of theterminal device 12 to display information that indicates a transfer ofthe operation authority (S03), and invalidates operations on the UIsection 30 of the terminal device 12 (S04). As the information thatindicates a transfer of the operation authority, information indicatingthat an operation on the UI section 30 of the terminal device 12 isinvalid is displayed on the UI section 30 of the terminal device 12, forexample.

Next, the remote operation section 36 notifies the image forming device10 of the transfer of the operation authority (S05). For example,information that indicates an instruction to transfer the operationauthority is transmitted from the terminal device 12 to the imageforming device 10 via the communication path N.

When information that indicates an instruction to transfer the operationauthority is received from the terminal device 12, the controller 24 ofthe image forming device 10 causes the UI section 20 of the imageforming device 10 to display information that indicates a transfer ofthe operation authority (S06). As the information that indicates atransfer of the operation authority, information indicating that anoperation on the UI section 20 of the image forming device 10 is validis displayed on the UI section 20 of the image forming device 10, forexample.

Next, the controller 24 of the image forming device 10 transmitsinformation indicating that an instruction to transfer the operationauthority has been received to the terminal device 12 via thecommunication path N (S07). Upon receiving the information, the remoteoperation section 36 of the terminal device 12 erases display of theinformation that indicates the transfer of the operation authority fromthe UI section 30 of the terminal device 12 (S08). That is, theinformation that indicates the transfer of the operation authority ishidden.

After that, a notification of the erasure of the display is sent fromthe terminal device 12 to the image forming device 10 (S09), and thecontroller 24 of the image forming device 10 erases the display of theinformation that indicates the transfer of the operation authority fromthe UI section 20 of the image forming device 10 (S10). That is, theinformation that indicates the transfer of the operation authority ishidden. In addition, the controller 24 of the image forming device 10cancels the invalidation of operations on the UI section 20 of the imageforming device 10 (S11). Consequently, operations on the UI section 20of the image forming device 10 are validated, and the UI section 20 mayreceive operations.

Similar processes are performed also in the case where the operationauthority is transferred from the image forming device 10 to theterminal device 12. That is, when a second specific sound (e.g. a voicesuch as “Operation is finished”) is detected by the sound detectionsection 22 of the image forming device 10 with the image forming device10 having the operation authority and with a specific reception screen(e.g. the function selection screen 40) displayed on the UI section 20of the image forming device 10, the operation authority is transferredfrom the image forming device 10 to the terminal device 12.Consequently, operations through the UI section 20 of the image formingdevice 10 are invalidated, and operations through the UI section 30 ofthe terminal device 12 are validated.

Screen transition in the image forming device 10 and the terminal device12 will be described in detail below.

First, screen transition for a case where the operation authority istransferred from the terminal device 12 to the image forming device 10will be described with reference to FIGS. 8A to 8C and 9A to 9C. FIGS.8A to 8C illustrate an example of screen transition in the terminaldevice 12. FIGS. 9A to 9C illustrate an example of screen transition inthe image forming device 10.

As illustrated in FIG. 9A, the function selection screen 40 illustratedin FIG. 5 is displayed on the UI section 20 of the image forming device10 as a specific reception screen, for example. In the remotecommunication state, as illustrated in FIG. 8A, a function selectionscreen 44 is displayed on the UI section 30 of the terminal device 12.The function selection screen 44 is the same screen as the screendisplayed on the image forming device 10. As a matter of course, ascreen prepared by customizing the function selection screen 40 for theterminal device 12 may be displayed on the terminal device 12. In thecase where the terminal device 12 has the operation authority,information indicating that the terminal device 12 has the operationauthority (e.g. a message saying “You have operation authority.”) isdisplayed on the function selection screen 44 as indicated by symbol 46.

When the user (e.g. an operator at the customer center) of the terminaldevice 12 utters a voice (e.g. a voice such as “Go ahead”, which is avoice corresponding to the first specific sound) for transferring theoperation authority to the image forming device 10 and the voice (firstspecific sound) is detected by the sound detection section 32 of theterminal device 12, the remote operation section 36 displays informationthat indicates a transfer of the operation authority (informationindicated by symbol 48) on the function selection screen 44 asillustrated in FIG. 8B. Examples of the information that indicates atransfer of the operation authority include display of a message saying“Operation authority will be transferred to user in communication.” orthe like. In addition, the remote operation section 36 invalidatesoperations on the UI section 30 of the terminal device 12. Consequently,operations on the UI section 30 of the terminal device 12 may not bereceived.

The remote operation section 36 notifies the image forming device 10 ofthe transfer of the operation authority, and erases the information thatindicates the transfer of the operation authority (information indicatedby symbol 48) from the function selection screen 44 as illustrated inFIG. 8C.

When a notification of the transfer of the operation authority isreceived from the terminal device 12, the controller 24 of the imageforming device 10 displays the information that indicates the transferof the operation authority (information indicated by symbol 50) on thefunction selection screen 40 as illustrated in FIG. 9B. Examples of theinformation that indicates the transfer of the operation authorityinclude display of a message saying “Operation authority has beentransferred to you. Please operate.” or the like. In addition, thecontroller 24 validates operations on the UI section 20 of the imageforming device 10. Consequently, operations on the UI section 20 of theimage forming device 10 may be received. The controller 24 displaysinformation indicating that the image forming device 10 has theoperation authority (information indicated by symbol 52) on the functionselection screen 40 as illustrated in FIG. 9C. Examples of theinformation indicating that the image forming device 10 has theoperation authority include display of a message saying “You haveoperation authority.” or the like.

Next, screen transition for a case where the operation authority istransferred from the image forming device 10 to the terminal device 12will be described with reference to FIGS. 10A to 10C and 11A to 11C.FIGS. 10A to 10C illustrate an example of screen transition in the imageforming device 10. FIGS. 11A to 11C illustrate an example of screentransition in the terminal device 12.

As illustrated in FIG. 10A, the function selection screen 40 isdisplayed on the UI section 20 of the image forming device 10, forexample. In the case where the image forming device 10 has the operationauthority, information indicating that the image forming device 10 hasthe operation authority (e.g. a message saying “You have operationauthority”) is displayed on the function selection screen 40 asindicated by symbol 54. In the remote communication state, asillustrated in FIG. 11A, the function selection screen 44 correspondingto the function selection screen 40 is displayed on the UI section 30 ofthe terminal device 12.

When the user (e.g. a customer) of the image forming device 10 utters avoice (e.g. a voice such as “Operation is finished”, which is a voicecorresponding to the second specific sound) for transferring theoperation authority to the operation authority 12 and the voice (secondspecific sound) is detected by the sound detection section 22 of theimage forming device 10, the controller 24 of the image forming device10 displays information that indicates a transfer of the operationauthority (information indicated by symbol 56) on the function selectionscreen 40 as illustrated in FIG. 10B. Examples of the information thatindicates a transfer of the operation authority include display of amessage saying “Operation authority will be delivered to operator.” orthe like. In addition, the controller 24 invalidates operations on theUI section 20 of the image forming device 10. Consequently, operationson the UI section 20 of the image forming device 10 may not be received.

The controller 24 of the image forming device 10 notifies the terminaldevice 12 of the transfer of the operation authority, and erases theinformation that indicates the transfer of the operation authority(information indicated by symbol 56) from the function selection screen40 as illustrated in FIG. 10C.

When a notification of the transfer of the operation authority isreceived from the image forming device 10, the remote operation section36 of the terminal device 12 displays the information that indicates thetransfer of the operation authority (information indicated by symbol 58)on the function selection screen 44 as illustrated in FIG. 11B. Examplesof the information that indicates the transfer of the operationauthority include display of a message saying “Operation authority hasbeen transferred to you.” or the like. In addition, the remote operationsection 36 validates operations on the UI section 30 of the terminaldevice 12. Consequently, operations on the UI section 30 of the terminaldevice 12 may be received. The remote operation section 36 displaysinformation indicating that the terminal device 12 has the operationauthority (information indicated by symbol 60) on the function selectionscreen 44 as illustrated in FIG. 11C. Examples of the informationindicating that the image forming device 10 has the operation authorityinclude display of a message saying “You have operation authority.” orthe like.

Next, screen transition for a case where the operation authority isforcibly transferred from the image forming device 10 to the terminaldevice 12 will be described with reference to FIGS. 11A to 11C and 12Ato 12C. FIGS. 12A to 12C illustrate an example of screen transition inthe image forming device 10.

As illustrated in FIG. 12A, the function selection screen 40 isdisplayed on the UI section 20 of the image forming device 10, forexample. In the case where the image forming device 10 has the operationauthority, information indicating that the image forming device 10 hasthe operation authority (e.g. a message saying “You have operationauthority.”) is displayed on the function selection screen 40 asindicated by symbol 62. In the remote communication state, asillustrated in FIG. 11A, the function selection screen 44 correspondingto the function selection screen 40 is displayed on the UI section 30 ofthe terminal device 12.

When the user (e.g. an operator at the customer center) of the terminaldevice 12 utters a voice (e.g. a voice such as “Operator will operate”,which is a voice corresponding to the third specific sound) for forciblytransferring the operation authority to the terminal device 12 and thevoice (third specific sound) is detected by the sound detection section32 of the terminal device 12, the remote operation section 36 transmitsinformation that indicates a request to forcibly acquire the operationauthority to the image forming device 10 via the communication path N.

When information that indicates a request to forcibly acquire theoperation authority is received from the terminal device 12, thecontroller 24 of the image forming device 10 displays information thatindicates a forcible transfer of the operation authority (informationindicated by symbol 64) on the function selection screen 40 asillustrated in FIG. 12B. Examples of the information that indicates aforcible transfer of the operation authority include display of amessage saying “Operation authority will be delivered to operator uponrequest from operator” or the like. In addition, the controller 24invalidates operations on the UI section 20 of the image forming device10. Consequently, operations on the UI section 20 of the image formingdevice 10 may not be received.

The controller 24 of the image forming device 10 notifies the terminaldevice 12 of the transfer of the operation authority, and erases theinformation that indicates the transfer of the operation authority(information indicated by symbol 64) from the function selection screen40 as illustrated in FIG. 12C.

When a notification of the transfer of the operation authority isreceived from the image forming device 10, the remote operation section36 of the terminal device 12 displays the information that indicates thetransfer of the operation authority (information indicated by symbol 58)on the function selection screen 44 as illustrated in FIG. 11B. Examplesof the information that indicates the transfer of the operationauthority include display of a message saying “Operation authority hasbeen transferred to you.” or the like. In addition, the remote operationsection 36 validates operations on the UI section 30 of the terminaldevice 12. Consequently, operations on the UI section 30 of the terminaldevice 12 may be received. The remote operation section 36 displaysinformation indicating that the terminal device 12 has the operationauthority (information indicated by symbol 60) on the function selectionscreen 44 as illustrated in FIG. 11C. Examples of the informationindicating that the terminal device 12 has the operation authorityinclude display of a message saying “You have operation authority.” orthe like.

In the exemplary embodiment, as has been described above, in the casewhere a voice corresponding to an instruction to transfer the operationauthority is detected, the operation authority is transferred betweenthe image forming device 10 and the terminal device 12 withoutperforming a specific operation other than providing a voiceinstruction. For example, in the case where a first specific sound isdetected by the terminal device 12 with the terminal device 12 havingthe operation authority and with the image forming device 10 not havingthe operation authority, the operation authority is transferred from theterminal device 12 to the image forming device 10 even if a user (e.g.an operator or a customer) does not perform a specific operation otherthan providing a voice instruction. Similarly, in the case where asecond specific sound is detected by the image forming device 10 withthe image forming device 10 having the operation authority and with theterminal device 12 not having the operation authority, the operationauthority is transferred from the image forming device 10 to theterminal device 12 even if a user (e.g. an operator or a customer) doesnot perform a specific operation other than providing a voiceinstruction. In addition, in the case where a third specific sound isdetected by the terminal device 12, the operation authority is forciblytransferred from the image forming device 10 to the terminal device 12even if a user does not perform a specific operation other thanproviding a voice instruction.

In the case where a first specific sound is detected by the terminaldevice 12 with a specific reception screen displayed on the UI section20 of the image forming device 10 when the terminal device 12 has theoperation authority and the image forming device 10 does not have theoperation authority, the operation authority is transferred from theterminal device 12 to the image forming device 10. In the case where thespecific reception screen is not displayed on the UI section 20 of theimage forming device 10, the operation authority is not transferred fromthe terminal device 12 to the image forming device 10 even in the casewhere a first specific sound is detected by the terminal device 12.Examples of the specific reception screen include the function selectionscreen 40, a password input screen, and a billing screen. On suchreception screens, it is assumed that a user (e.g. a customer) of theimage forming device 10 himself/herself performs an operation, and thata remote operation by a user (e.g. an operator at the customer center)of the terminal device 12 is not suitable. That is, it is assumed thatthe user of the image forming device 10 himself/herself is required toperform an operation. By transferring the operation authority from theterminal device 12 to the image forming device 10 in the case where afirst specific sound is detected by the terminal device 12 when such aspecific reception screen is displayed on the image forming device 10,the operation authority for a screen, on which an operation by the userof the image forming device 10 himself/herself is required, istransferred to the image forming device 10. Meanwhile, if the operationauthority is transferred from the terminal device 12 to the imageforming device 10 in the case where a first specific sound is detectedby the terminal device 12 even when a specific reception screen is notdisplayed on the image forming device 10, the operation authority for ascreen, on which an operation by the user of the image forming device 10himself/herself is not suitable, (e.g. a screen, on which an operationby an operator is suitable) may be transferred to the image formingdevice 10. That is, the operation authority for a screen, on which anoperation by an operator is suitable, or a screen, on which an operationshould be performed by an operator, may be erroneously transferred tothe image forming device 10. In the exemplary embodiment, the operationauthority is not transferred to the image forming device 10 when aspecific reception screen is not displayed on the image forming device10.

The image forming system according to the exemplary embodiment isassumed to be used in the following scenes. For example, in the casewhere a user (e.g. a customer) of the image forming device 10 does notknow how to operate the image forming device 10, it is assumed that thecustomer consults an operator at the customer center to receive aninstruction. In this case, the operator remotely operates the imageforming device 10 through the terminal device 12 to cause the imageforming device 10 to display a screen (e.g. a function selection screen,a password input screen, and a billing screen) requested by thecustomer, and transfers the operation authority from the terminal device12 to the image forming device 10 when the screen is displayed.Consequently, an operation performed by the customer on the imageforming device 10 is validated. For example, the customer selects aspecific function on the function selection screen, inputs a password onthe password input screen, or performs an operation that is necessaryfor a billing process (by way of example, presses a button to execute aprocess corresponding to the billing) on the billing screen. In the casewhere the operation is completed, the operation authority is transferredfrom the image forming device 10 to the terminal device 12, and theoperator performs subsequent operations (e.g. an operation that istroublesome to the customer, a complicated operation, and an operationthat should be performed by the operator). In the exemplary embodiment,the operation authority is transferred between the image forming device10 and the terminal device 12 without performing a specific operation(e.g. a button operation) other than providing a voice instruction insuch scenes.

In the exemplary embodiment described above, an operation is invalid,that is, an operation may not be received, on the UI section of thedevice not having the operation authority. In another example, anoperation may be valid, that is, an operation may be received, also onthe UI section of the device not having the operation authority. In thiscase, information indicating that an operation is prohibited (e.g. amessage saying “Do not operate”) may be displayed on the UI section ofthe device not having the operation authority so that an operation isnot performed on the UI section of the device.

Images that represent software keys that represent the hardware keysprovided on the UI section 20 of the image forming device 10 may bedisplayed on the UI section 30 of the terminal device 12, and the remoteoperation section 36 of the terminal device 12 may have a function ofremotely operating the hardware keys of the image forming device 10. Inthis case, an operation on the hardware keys on the UI section of thedevice having the operation authority is valid, and an operation on thehardware keys on the UI section of the device not having the operationauthority is invalid.

In the example described above, the image forming system that includesthe image forming device 10 and the terminal device 12 has beendescribed above. However, the process related to the transfer of theoperation authority through a voice instruction may be applied tosystems that include a device other than the image forming device 10.

Each of the image forming device 10 and the terminal device 12 isimplemented through cooperation between hardware resources and software,by way of example. Specifically, each of the image forming device 10 andthe terminal device 12 includes one or more processors such as centralprocessing units (CPUs) (not illustrated). The one or more processorsread and execute a program stored in a storage device (not illustrated)to implement the functions of the various components of the imageforming device 10 and the terminal device 12. The program is stored inthe storage device by way of a storage medium such as a CD and a DVD, orby way of a communication path such as a network. In another example,the various components of each of the image forming device 10 and theterminal device 12 may be implemented by hardware resources such as aprocessor and an electronic circuit. For implementation, a device suchas a memory may be utilized. In another example, the various componentsof each of the image forming device 10 and the terminal device 12 may beimplemented by a digital signal processor (DSP) and a field programmablegate array (FPGA).

The foregoing description of the exemplary embodiment of the presentinvention has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit theinvention to the precise forms disclosed. Obviously, many modificationsand variations will be apparent to practitioners skilled in the art. Theembodiment was chosen and described in order to best explain theprinciples of the invention and its practical applications, therebyenabling others skilled in the art to understand the invention forvarious embodiments and with the various modifications as are suited tothe particular use contemplated. It is intended that the scope of theinvention be defined by the following claims and their equivalents.

What is claimed is:
 1. An information processing apparatus comprising: adetection unit that detects a sound indicating a transfer of operationauthority; an operation reception unit that receives an operation by auser; and one or more processors configured to act as a remote operationunit that: (i) operates an external device, at least one of operationson the external device being invalidated, (ii) validates an invalidatedoperation on the external device when the sound is detected with areception screen for receiving a specific input by the user displayed onthe external device, and (iii) invalidates the operation reception unitand causes a display to display information indicating that theoperation reception unit is invalidated when the sound is detected bythe detection unit with the reception screen displayed on the externaldevice.
 2. The information processing apparatus according to claim 1,wherein the remote operation unit causes the display to displayinformation indicating that the operation reception unit is invalidatedwhen the operation reception unit has received an operation.
 3. Theinformation processing apparatus according to claim 1, wherein theremote operation unit causes the display to display informationindicating that the operation reception unit is validated when anothersound is detected by the external device with information indicatingthat the operation reception unit is invalidated displayed on thedisplay.
 4. The information processing apparatus according to claim 1,wherein when the reception screen is not displayed on the externaldevice, the remote operation unit does not validate an operation on theexternal device, which has been invalidated, even when the sound isdetected by the detection unit.
 5. The information processing apparatusaccording to claim 1, wherein the remote operation unit validates theoperation reception unit and invalidates an operation on the externaldevice, which has been validated, when another sound is detected by thedetection unit with an operation on the external device validated andwith the operation reception unit invalidated.
 6. A non-transitorycomputer readable medium storing a program causing a computer tofunction as: a remote operation unit that: (i) operates an externaldevice, at least one of operations on the external device beinginvalidated, (ii) validates at least one of the invalidated operationson the external device when a sound indicating a transfer of operationauthority is detected with a reception screen for receiving a specificinput by a user displayed on the external device, and (iii) invalidatesan operation reception unit, which receives an operation by the user,and causes a display to display information indicating that theoperation reception unit is invalidated when the sound is detected bythe detection unit with the reception screen displayed on the externaldevice.